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ABSTRACT 

Background Documentation of the long-term impact of 
supportive supervision using a monitoring tool in STI 
intervention with sex workers, men who have sex with 
men and injection-drug users is limited. The authors 
report methods and results of continued quality 
monitoring in a large-scale STI services provided as 
a part of a broader HIV-prevention package in six Indian 
states under Avahan, the India AIDS Initiative. 
Methodology Guidelines and standards for STI services, 
and a supportive supervisory tool to monitor the quality 
were developed for providing technical support to STI 
component of large-scale HIV-prevention intervention 
through 372 project-supported STI clinics. The tool 
contained 80 questions to track the quality of STI 
services provided on a five-point scoring scale in five 
performance areas: coverage, quality of clinic and 
services, referral networks, community involvement and 
technical support. 

Results The tool was applied to different STI clinics 
during supportive supervision visits conducted once in 
every 3 months to assess quality, give immediate 
feedback and develop a quality score. A total of 292 
clinics managed by seven lead implementing partners in 
six Indian states were covered in 15 quarters over 
45 months. Overall quality indicators for the five 
performance areas showed a three- to sevenfold 
improvement over the period. 
Conclusion It was possible to improve quality over the 
long-term in STI interventions for sex workers, men who 
have sex with men and injection-drug users using an 
interactive and comprehensive supportive supervision 
tool which gives on-the-spot feedback. However, such 
an effort is time-consuming and resource-intensive, and 
needs a structured approach. 



BACKGROUND 

The control of STIs among high-risk groups such as 
sex workers (SWs), men who have sex with men 
and transgenders (MSMs/TGs), and injecting-drug 
users (IDUs) is an important strategy for 
containing HIV infection. 1 It is a challenge to 
provide acceptable, accessible and high-quality 
sexual-health services to control and treat STIs 
among high-risk groups. 2 Establishing standards 
against which quality can be measured (quality 
assurance) and developing approaches to solve the 



problems affecting the quality (quality improve- 
ment) are recognised as two important means to 
ensure high-quality services. 3 4 Several approaches 
to evaluate and enhance the quality of STI services 
are reported using different methodologies such as 
mystery patients, record reviews, patient exit 
interviews, management tools and supervisory 
measures. 5-10 The managerial approach of 
supportive supervision involving joint problem- 
solving, record review and observing clinical practice 
has consistently shown a moderate to marked 
effect in quality improvement. 5 11 12 However, the 
reported experience of using these various 
methodologies to measure quality was a one-time 
assessment rather than pursuing measures to 
improve quality over a long period. 11 13-16 In addi- 
tion, most reports employ methodologies to assess 
a single aspect of service delivery, from a clinical, 
managerial or community perspective rather than 
combined assessments. 17 In this paper we report on 
the experience and results of 4 years of work to 
monitor and improve STI service quality of high- 
risk groups using a supportive supervision tool that 
incorporated community, clinical and management 
perspectives. 

METHODS 

Avahan, the India AIDS Initiative, has supported 
the HIV-intervention programme in six HIV high- 
prevalence states in India through various grants 
awarded beginning from December 2003 to August 
2005. As of December 2007, the programme 
provided HIV-prevention services to about 280 000 
SWs and MSMs through a pyramidal virtual 
organisational structure consisting of a central STI 
team, seven lead implementing partners who 
subgranted to over 140 local non-governmental 
organisations (NGOs). 18 The services provided 
through project supported clinics, peer educators 
and out-reach workers included STI and primary 
HIV care, condom provision, behaviour change 
communication and community mobilisation to 
build capacity for community ownerships. The STI 
clinic services were scaled up, starting from 
December 2004. Details of early scale-up are avail- 
able in a previous publication. 19 

The delivery of STI services worked at three 
levels. 19 At the grass-root level, NGOs organised 
services to SWs, MSM/TGs and IDUs through 
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designated clinic settings such as static clinics, mobile clinics in 
a vehicle and health camps. Some static clinics were located at 
private general practitioner clinics who were identified by the 
community as preferred service providers. At the middle level, the 
seven state lead implementing partners provided technical and 
management support, established a supervisory system and gave 
logistical assistance to facilitate overall STI service delivery. At 
the central level, an STI capacity building team was responsible 
for ensuring high quality and standardised STI services which 
were provided uniformly across the seven lead partners and their 
subgrantees. Clinical operational guidelines, standard operating 
procedures and corresponding supervisory handbooks were 
developed to support standardisation and supervision. 20 21 The 
central team conducted trainings, provided support and 
mentoring to mid-level supervisors, monitored overall activities 
and readjusted guidelines based on the experiences, monitoring 
data and operation research. Overall, the capacity-building 
system was responsible to facilitate the delivery of uniform high- 
quality services for STIs management. Counselling and basic HIV 
management services were added later. 

The supervisory system was two-tiered: a state and 
a central level. The state-level supervisory team consisted of 
staff from the lead implementing partner who conducted 
systematic, routine periodic visits to the designated clinics 
supported and managed by them. Every clinic was covered by 
these visits. The supervisory visit schedules could be adjusted 
to respond to the need for state supervisors based on the 
requirements of individual clinic. Frequent visits were given to 
clinics which needed more support for quality improvement 
such as newly started clinics and those with new staff. The 
central STI capacity-building team conducted 'dipstick' 
supervisory visits to different clinics supported by each lead 
implementing partner once in every 3 months. Dipstick super- 
vision consisted a visit by central supervisors accompanied by 
state supervisors to preselected clinics. Around 10% of existing 
clinics under each lead implementing partner were selected in 
each quarter in consultation with state clinic supervisors. 
Different clinic clusters were visited by the central team in each 
quarter. Clinic staff were aware of supervisory visit plans 
beforehand at least a week earlier. 

The central STI capacity-building team developed a clinic 
quality monitoring tool in a participatory manner as a compo- 
nent of regular supervision, for which guidelines were outlined 
in the supervisory handbook. 21 This tool was used as part of 
regular supervision and assisted in monitoring the quality of 
clinical services against the prescribed standards presented in 
Clinic Operational Guidelines and Standards. 19 20 The purpose of 
the tool was twofold: (1) to serve as a checklist to help clinic 
supervisors from the state lead implementing partners to 
support, monitor and improve the overall quality of service 
delivery during their periodic visits; and (2) to track quality of 
STI services over time. 

This tool assessed five performance areas of STI clinical 
services, as shown in the table 1. The coverage performance area 
measured accessibility, acceptability and contact coverage, which 
were the three important domains related to provision of STI 
services in programme settings where exclusive services are made 
available to SWs, MSM/TGs and IDUs. 22 23 The coverage 
component of the tool was devised to explore various structural 
and environmental barriers for service uptake such as geograph- 
ical distance, mobility, work limitations and social stigma, so 
that actions could be taken to enhance the coverage 24 The 
quality of clinic and services performance area measured 10 clinic 
and service components (see table 1) such as correct treatment, 



counselling, infection control, confidentiality, drug stock and 
record keeping measured against the defined standards. 20 The 
referral network measured the availability of adequate referral 
network and the use of STI clinics as an entry point for HIV 
testing and treatment services. The community involvement 
performance indicator measured the involvement of the 
community in the clinic service delivery, which is an explicit 
component of the programme to facilitate client-orientated 
services, community ownership and sustainability 25 The tech- 
nical support performance indicator measured the adequacy of 
supportive supervision provided to the clinic staff by the lead 
implementing partner level technical and management team. 
The tool documents the observations in yes/no or numerical 
form (percentage and numbers) for around 80 observations 
including interviews, clinical observations, record reviews and 
data analysis. The yes/no questions were scored 0 or 1. Numerical 
answers were converted to an ordinal score between 0 and 5. 
Based on the number and type of questions under each enquiry 
area, a mean score between 0 and 5 for each of the five enquiry 
areas could be calculated. For example, coverage component had 
five numerical questions (table 1). Based on percentage of 
coverage, each question was given a score between 0 and 5. The 
mean score for these five questions was calculated to obtain the 
score for coverage. In a similar manner, the mean score between 
0 and 5 for each of the five enquiry areas was calculated using the 
supervisory tool. Some of the subcomponents of the five main 
performance indicators in the tool have undergone refinement 
over the period based on field experiences. Such adjustments were 
mostly improvements in the definition of subcomponents and its 
measurement methodologies so that the observations could be 
documented quantitatively. The clinic quality monitoring tool is 
a participatory tool which guides the supervisors through 
a systematic supportive supervision session requiring about 
2—3 h per clinic assessment with the active involvement of the 
service providers. 

During the supervisory visits, the central team documented 
the observations on the clinic quality-monitoring tool with 
active participation of the clinic staff. Two outcomes were 
generated from the tool. First, the central STI capacity building 
team generated a technical report and recommendations based 
on the observations leading to immediate follow-up actions by 
the clinic to improve the service quality. The state supervisory 
team followed up the corrective actions during their subsequent 
visits. The second, clinic quality-monitoring tool and technical 
reports were periodically reviewed by central STI capacity 
building team to generate an objective report using an ordinal 
score of 0—5 for each of the above five enquiry areas at the clinic 
level. For each of seven state lead implementing partners, quar- 
terly clinic quality scores were generated under five performance 
indicators. The scores were calculated by a central STI capacity 
building team by averaging the scores for all clinics visited under 
the respective partner in that quarter. Final quarterly quality 
scores for five performance indicators were obtained by averaging 
the score for all the seven state lead partners. Whenever a visit to 
a state lead implementing partner was not conducted in 
a particular quarter, the clinic quality score from the previous 
quarter was carried forward, assuming no change in the quality 
in that quarter. The statistical significance of the quality score 
trend over the time was assessed based on the correlation coef- 
ficient using SPSS (SPSS, Chicago). 

RESULTS 

The total number of static project owned clinics managed 
by the seven state lead partners under Avahan was around 
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Table 1 Summary of the clinic quality-monitoring tool used to measure the quality against defined standards 21 


Question (Indicator) 


Main components of composite scale 


Methodology 


1 . Are the services accepted and 


^ Clinic service uptake 


Quantitative analysis of clinic and 


accessed? (Coverage) 


^ Clinic orientation to sex workers 


programme data (calculated in 




^ Clinic orientation to provide STI 


percentage) 




services 






► Monthly check-ups — uptake 






► Presumptive treatment for 






STIs uptake 




2. Whether adequate clinical care, 


► Clinic staffing and training 


Provider interview, provider observation 


effective drugs and preventive support 


(five subcomponents) 


and review of records selected randomly 


were given? (Quality of clinic and services) 


► Clinic set-up (five subcomponents) 






► Performance at clinic encounter 






(five subcomponents) 






► Performance on counselling (five 






subcomponents) 






► Performance on correct treatment 






(review of 10 randomly selected 






records from previous 3 months) 






► Infection control and waste 






management (five subcomponents) 






► Drug and condom supply 






(five subcomponents) 






► Ethical standards and confidentiality 






(five subcomponents) 






^ Completeness of patient records 






(review of 10 randomly selected 
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^ Clinic documentation and roporting 






(five subcomponents) 




J. Is the clinic an entry point tor prevention 


^ Availability of referral directory 


Providers interview and record review 


to care continuum? (Referral network) 


► Documentation of referrals 






► Formal linkages with HIV testing 






► Formal linkage with other referrals 






► Periodic meeting with referral sites 




4. Are MSMs and SWs taking active role 


► Drop-in centre utilisation 


Staff interview and clinic observations 


in managing and delivering the services? 


► Involvement of community members in 




(Community involvement) 


clinic activities 






^ Involvement of community in STI 






follow-up in the field 






^ Periodic meeting between clinic and 






outreach staff 






► Recommendations of meetings are 






acted upon 




5. Are we providing adequate supportive 


► Availability of designated supervisor 


Review of scale-up information, field 


supervision to the clinics? 


► Frequency of clinic visit by supervisor 


observations, interviews and consensus 


(Technical support) 


► Overall clinic load per supervisor 


score 




► Use of supervisory tool 






► Joint action plan implementation 





372 by October 2008 compared with 106 in February 2005 
(table 2). During the past 45 months, the central support team 
had made 15 rounds of visits to state lead partners to cover 
292 clinics and provided onsite support to clinic staff and state 
supervisory team. The scoring exercise was carried out by the 
central team for all quarters except the 2nd, 3rd, 5th and 14th 
quarters. On 14 occasions, the visit to one or more state lead 
partners was not made in a particular round, and the clinic 
quality score from previous quarter was carried forward, 
assuming no change in the quality in that quarter. Thus, the 
analysis presented here includes 292 clinic visits conducted in 
11 rounds in 4 years. 

All five performance indicators of STI clinic services showed 
consistent improvement over the 45 months of observation, as 
summarised in the table 2. The performance indicator with the 
largest increase was the coverage component which increased 
sevenfold over a period of 45 months, to reach 3.24. The quality 
of clinic and services attained the score of 4.27 followed by 
community involvement at 4.21, technical support at 4.14 and 
referral networks 3.80 by October 2008. 

The time-trend analysis of the overall mean score for all five 
performance indicators showed a threefold increase in the score 



in three phases of progress (figure 1). The quality score 
improvement trend is statistically significant with a correlation 
coefficient of 0.88 (p<0.001). The first year beginning April 2005 
had an accelerated quality improvement; the second and third 
years (April 2006— January 2008) showed a slow but continued to 
increase, whereas in the fourth year (April 2008) onwards, it 
appears to have stabilised. 

DISCUSSION 

We have shown that quality improvement in STI clinic services 
is an important component of a large-scale STI intervention for 
high-risk groups. In addition, a supervision-monitoring tool 
could be used simultaneously for real-time quality improvement. 
The clinic quality monitoring tool was used in a large-scale HIV/ 
STI-prevention programme setting among SWs, MSM/TGs and 
IDUs in six Indian states over a period of 45 months to monitor 
and document quality improvement in various aspects of STI 
services. 

This experience reaffirmed the usefulness of supportive 
supervision in improving the quality of clinical services. 5 11 26-28 
For example, primary healthcare operational research in 12 
countries concluded that supervisory system is an essential 
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component of quality improvement. The experience also 
confirmed the value of a locally appropriate tool for structured 
supervision. Methodical supervision using checklists has resulted 
in quality improvement in practice of primary health treatment 
guidelines in Nepal and diarrhoea management in Nigeria. 26 In 
the Philippines, the use of a supervisory tool improved indicators 
in a primary health facility by 42 per cent in the experimental 
group compared with 18% in control group. 30 

There are three aspects of the quality-monitoring tool used in 
this report that address some of the shortcomings in previous 
publications. First, most reports on the impact of supervision 
noted effects after a short period of observations or as pilot 
studies. 11 26 In the experience reported here, dipstick supervision 
by an external team over a long period coupled with ongoing 
supervision and technical support by the organisation respon- 
sible demonstrated short-term rapid gains in quality and long- 
term maintenance. Second, this tool covers three aspects of 
quality improvement: client or patient, managerial and clin- 
ical. 17 Other studies have measured quality improvement of 
services through client satisfaction surveys, whereas the Avahan 
programme focussed on community involvement in design and 
implementation of STI services. 18 The community involvement 
approach used in Avahan takes the client perspective further to 
engage the client in clinical service quality improvement. 
Community hubs, called 'drop-in centres' usually attached to 
the clinics, were established to provide a space for community 
interaction. The set-up is intended to progressively involve 
community in clinic activities such as receptionists, clinical 
assistance, infection control management and conducting exit 
interviews to provide feedback on community perceptions on 
clinic services and management. In addition, community 
involvement has a larger dimension in making the HIV/STI 
intervention more effective. 31 Finally, this tool has attempted to 
monitor service coverage based on the framework of an effective 
coverage in health systems described by Tanahashi. 22 This 
coverage model envisages five domains: availability, accessi- 
bility, acceptability, contact and effective coverage, and recom- 
mends managerial decisions and health policies needed to 
enhance the coverage in all these domains. 22 23 The clinic quality 
monitoring tool reported here addressed issues related to 
accessibility, acceptability and contact coverage in a setting 
where STI services were made available to the high-risk and 
marginalised populations. Further, it explored various barriers 
for providing quality STI services and attempted to resolve 
them. 24 

There were several limitations in the study. The clinics 
visited by central team at any quarter were not representa- 
tive of all existing Avahan clinics due to possible selection 
bias, as they were selected in consultation with state lead 
implementing partners. This bias might have directed the 
visits to better performing clinics. However, the central team 
visited most of the clinics over the period limiting the bias 
over the long term. On 14 occasions (14/77=18%) the visit to 
a state lead partner was not made in a particular round; the 
quality performance score from the previous round was carried 
forward, assuming no change in that quarter, which might have 
underestimated the overall performance score. On the other 
hand, the fact that clinic staffs were aware about the supervisory 
visits may have bolstered the quality in two general ways. First, 
there could be a conscious effort made by clinic staff to display 
better quality, and second, there could be an improvement in the 
performance just because the staff were being observed. 32 
However, as patient records for review were selected randomly 
from the previous 3 months, such overestimation is limited. This 
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Figure 1 Quality of overall STI clinic 
services measured from 292 clinic visits 
using clinic quality monitoring tool in 
the scale 0—5. 
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tool was developed for Avahan-supported static clinics and not 
suitable for mobile clinics, health camps and other public-funded 
clinics. It was difficult to apply this tool in a busy private practice 
or other preferred service provider settings due to their time 
constraints. External technical support has limited influence over 
the service delivery of private practitioners. Modification of the 
tool for different settings is necessary. It is recognised that this 
tool is too time-consuming to be utilised in routine supervision 
by the national AIDS control programme. Going forward, the 
key elements of the supervision tool will be incorporated into 
a simpler supervision and monitoring tool. 

In summary, the experience of long-term use of a clinic 
quality monitoring tool for supportive supervision and 
monitoring has shown that clinical services in Avahan had 
improved and sustained quality in all five aspects of clinic 
performance (coverage, quality, referral networks, community 
involvement and technical support). The experience demon- 
strated the usefulness of the participatory monitoring tool to 
track and provide quality improvement inputs in real time 
over a long period that facilitated immediate feedback at the 
time of supervision. Such a comprehensive monitoring and 
quality improvement effort in the STI programme could be 
resource-intensive and time-consuming, and needs a structured 
approach. Cost-effectiveness, impact of reduced comprehen- 
siveness of supervision areas and replicability of such an 
approach in limited resource settings need to be the focus of 
future work. 



Key messages 



► Continued quality improvement over a long period is possible in 
sexually transmitted infection interventions for high-risk 
groups. 

► Locally appropriate supportive supervision tool is important to 
ensure quality improvement in clinics providing STI services. 



► Combined clinician, patient and managerial perspectives are 
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